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A BST^ ACl 

Theorizing atout evaluation should b» conceptualized 
as a hu«an activity governed by certain strategies and principles. 
The ^-heories advanced ty various evaluators have changed over the 
years, thus illustrating ten principles of evaluation. The starting 
po^nt for theory development cr iro.dif ica ticn is self- reflection and 
r«v-;e« cf one's owi writings. Public conceptims cf a theory are 
ba4d uFCn publications describing it; different conceptions are 
possible. Theoretical formulations are subject to continual change. 
Other principUs suggest major factors for modifying ^'fl^^^Jf"^^ 
theory. *t conference sumposia, for example, an evaluation theory can 
be interpreted by colleagues, and compared and contrasted to other 
^-h«ori€s; here also, evaluators can learn tc understand contrasting 
ariuments and to challenge or extend inferences. Another phase in 
evaluation theory development becoies possible when theory 
consolidation or theory categorization takes place. Other factors 
influencing theory development include fi^ld based experience, and 
interaction with colleagues who have compatible points of view. 
Finally, evaluation theory development is conditioned by human temgs 
and their associations, mctivations, intuiticrs, and imaginations. 
(GDC) 
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The Human Activity of Evaluation Theorizing 



Marvin C. AT kin 
Frederick Ellett, Jr.* 
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The literature is replete with numerous attempts at developing so- 
called "theories" of evaluation, with applications of these theories, and 
critiques or comparisons of them. This paper is not concerned with exam- 
ining or criticizing an evaluation theory or even groups of theories. 
Rather, the focus of this paper is a consideration of various strategies 
th.t people properly follow when they are engaged in evaluation theorizing. 

WHY EXAMINE EVALUATION THEORIZING? 
Why is it important to engage in the activity of examining evalua- 
tion theorizing? We feel that the primary benefit to be gained from 
careful analysis of evaluation theorizing, fier se, will be the signifi- 
cant insights provided into the nature of evaluation theory development. 
Through such increased understanding of theorizing, a theorist will 
acquire a heightened ability to personally engage in theory construc- 
tion more reflectively, carefully, and probably more successfully. A 
further benefit, derived from an inquiry into evaluation theorizing, is 
a better understanding of the nature of evaluation theory generally a:.d 
of the various evaluation theories. 

An example may help to illustrate this general point. While pev^ons 
developing a theory usually base their formulation on tentative conjectures, 
5f7Ri-Authors sh£ j:int and equal responsibility for this paper. 



others reading the theory may have the impression that certain issues 
are quite settled. The tentativeness is usually lost to readers who may 
not regard the issue in that way. An analysis of evaluation theorizing 
draws attention to the process by which theories evolve, and as a result 
it calls attention the tentativeness of most evaluation theory statements. 
And here we refer to the tentativeness as felt by the theorist himself. 

What we set as our task in this paper is, then, to draw out some 
of the special features of the human activity of evaluation theorizing. 
It must be stressed, of course, that our work is only a first step. 
Like any activity, this activity of evaluation theorizing can be eluci- 
dated in terns of its ends. In this case the end is a definable theory 
of evaluation.'' It is our contention that, in carrying out the process 
of evaluation theory building, there are several strategies and princi- 
ples which are useful in guiding one's theorizing. Our essay here is 
a step towards providing a comprehensive examination of those strategies 
and principles. 

Part of the data base for the examination of the strategies and 
principles of theorizing is to be found in our experiences and observa- 
tion of the evolution of various evaluation theoretic positions. In 
part, some of our findings are derived from the comments of evaluation 
theorists who participated in a 1978 AERA Symposium Session. In that 



Hhere are many forms of theories and different kinds of theory 
For example, there are both descriptive and normative evaluation 
theories, fhu'i one shouldn't suppose without further argument 
that all evaluation theorists are pursuing the same ends, it 
could be that they are talking about different aspects of eval- 
uation; it could also be -.hat only the shared use of the tenn 
'evaluation' ties them together. 



session a set of papers were presented in which some persons "recon- 
structed" the views of Egon Guba, Robert Rippey and Robert Wolf, and 
posited the change of evaluation views that those theorists might have 
had as a consequence of reading a particular philosophical treatise, 
(The New Rhetoric by Perelman and Olbrechts-Tyteca). The theorists 
themselves provided responses to the papers. Many of the strategies 

and principles we will mention today were drawn from the discussion 
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among these evaluation theorists and role players. 

When we refer to sets of strategies and principles which comprise 

evalua:ion theorizing we do not mean to imply a system akin to formal 

logic. Informal logic might be more fitting; perhaps the set should 

simply be called the canons of rational theorizing. Given that the 

end is to develop a justifiable theory, a person will find that certain 
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principles and strategies offer rational guidance towards it. 

Let us begin with a very general observation. During the various 
stages of the activity, a person's theory is best thought of as a loosely 
interwoven collection of beliefs, assumptions, expectations, aspirations 
attachments, obligations, preferences and perceptions. Even at the point 
that ideas are written down, fluidity and tentativeness are more charac- 
teristic than the rigidity and fixedness that one associates with a 
system of formal logic (or with a well -tested empirical theory.) 



^It is our contention that theorists, in a sense, are familiar _ 
with the strategies and principles. Our task is to make what is 
implicit, explicit, so that they can be more effective in guiding 
theorizing. 

^We do not mean to imply from all of the preceding discussion that 
theorizing is a purely intellectual activity. The disposition to 
consider criticism and the readiness to change one's theory when 
the evidence demands it are important to the activity. Courage, 
openness, c..id even persistence may well be important virtues here. 



Furthermore, though two theorists may share the same beliefs, and percep- 
tions, they may well disagree elsewhere. And it is important to notice 
such disagreements. 

(a) One may regard a matter as resolved , the other as 
tentative , with respect to certain standards of 
evidence and argumentation. 

(b) One may regard a matter as essential , the other as 
peripheral . 

(c) One may hold that a certain kind of relationship 
exists between the ideas; the other may hold 
another kind. 

In general, unless explicit care is taken, the distinction between 
"tentative" and "resolved" and the distinction between "essential" and 
"peripheral" are left unclarified. Without a strenuous effort at recon- 
struction, the reader is likely to suppose that all the beliefs and per- 
ceptions expressed by the writer are fixed, resolved, and quite essential. 
This supposition is likely to misconstrue the theorist's position in a 
significant way. 

The Role of Writing 

As we have said earlier, a writer's theory is a loosely interv/oven 
system of beliefs and perceptions about evaluation. What a writer be- 
lieves or perceives to be his theory of evaluation is in a real sense 
somewhat unknown to him. For it is extremely difficult to organize, 
systematize, substantiate, and to criticize one's sy stein of beliefs and 
perceptions. In fact, it is the written statement of a theory which is 
best used for examination and reformulation of theoretical positions. 



Accordingly, it is recommended that a serious writer should arrange 
to get his beliefs and perceptions down on paper. He should strive to 
formulate the key ideas, directions, and problems, giving as much of 
the rationale from his beliefs as he can. Of course, this working note- 
book will prepare the writer to communicate his beliefs and perceptions 
to other writers. Here we want to point out that the working note-book 
will enable a writer himself to get better control of his beliefs and 
perceptions. Thereby, he will be in a better position to understand and 
to criticize his own theorizing. In so doing, he will be engaged in 
justifiable theorizing. 

A plausible starting point for evaluation theory development requires 
that theorists gain adequate control of their own written views.^ When 
one first tries to understand a theory it is necessary to stay as close 
as possible to the way it is written-in fact, the first examination of 
a theoretic position consists of attempts at paraphrasing and character- 
ising the written views. Now, if it is one's own theory that is being 
examined, the starting point is self reflection or review of previous 
writings; people often don't have control of the central themes of their 
own views. This then, is the first principle: 

Principle 1 : The starting point for t heory development or 
modification is the examination by t he Theorist of the 
formulations of that theory. 

^The reason that this is only a plausible or possible starting point 
is that theoreticians may start with a problem or hypothesis which 
is apparently unrelated and only later on work back to relating the 
hypothesis or the new idea to the theoretic position. 



Various Perceptions 

A writer's public statements and papers are likely to be perceived by 
different people in different ways. The general public, say, may perceive 
them in one way, while the writer perceives them in another. (Needless to 
say, students often have different perceptions from both of these.) 

It may not be supposed that the writer is always the ultimate 
authority on what his public pronouncements say. In some cases his public 
statements and papers may express a belief or perception which he never 
intended to express. Continued differences of interpretations of his 
writings may well lead him to reconsider his statements and papers to 
see if he had misstated his position. 

Often a writer is interested in making sure that different groups 
of people understand at least in rough ways what his position is so that 
such people may accept it.^ But, the writer can also use such a public 
understanding as a basis for critical review of his theory. If he is com- 
mitted to espousing his perceptions and beliefs to just criticism, then, 
as a first step he must help the reader to understand it adequately. Many 
opportunities for serious, intellectual discussion abort because the part- 
icipants have failed to understand one another. This leads us to Principle 2, 
Principle 2 : A public conception of a theory is based primarily 
upon publications describing that theor y, but different concep- 
tions are possible . 

^it is important to note that we have characterized the writer's 
theory as what he believes or perceives it to be. When we talk 
about a theoretical position of someone, we are talking about 
the beliefs and perceptions of relationships that he holds. 
Ultimately, of course, the person's beliefs, perceptions and 
perceived relations must be submitted to rational defense. 
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This principle was clearly illustrated in the previously mentioned 
symposium. The several persons charged with the task of reconstructing 
the theoretic positions of Guba, Rippey and Wolf, become thoroughly 
"familiar" with the writings of "their" theorist. In fact, the papers 
(Alkin, 1979) were strong attestations that, for these writers, the 
theorists were represented in total by their written formulations. What 
is regarded as the person's theory is often the common interpretation 
of the meaning of the written statement. In fact, such a view of the 
theory may not express the intentions of the theorist at the time of 
writing nor his current views. ^ 

The symposium papers provided numerous examples of the tendency of 
readers to view a theorist's position in terms of his writings. In fact, 
theorists are usually personified by their own written words. Richard 
Daillak in discussing the utility of Wolf's judicial evaluation model 
said, "I have often commented on the need to include human judgement as 
a source of evidence in evaluation and policy decision making (e.g. Wolf, 
1973, 1975)..." Coming to see the other person's point of view is often 
accomplished by taking on his point of view. 

Elaine Lindheim relied on Egon Cuba's published work to reconstruct 
the educator and his confrontation with the new Rhetoric (TNR): "My 
basic definition of evaluation has not changed since reading this book. 

^Interestingly, the one person who had opportunity to enter into 
personal discussion with a theorist prior to writing the paper 
was perceived by that theorist as being too firmly rooted in 
his written statements and not sufficiently cognizant of his 
current actual position. 



Evaluation is the process of obtaining and providing useful information 
for making educational decisions. (My 1970 article, "Evaluation During 
Development,".... states this definition.)" She further states, in 
taking Guba's point of view, "my evaluation model focuses upon the 
decision-maker as the central determinant in why and how any information 
is gathered..." What she regards as the view of Egon Guba turns out to 
be an historical artifact, for the theorist himself responds, "I am not 
the 'real' Egon Guba. That Egon Guba was the one working and writing 
seven to 10 years ago... The real Egon Guba of 1978... is really quite 
different from what Elaine has suggested."^ 

The Role of Time 

The fact that conceptions of theoretic positions are viewed as fixed 
in the written word, provides us with a serious problem. For, as we have 
demonstrated in the previous example, theoretical viewpoints are not fixed 
and are ever changing. The problem evolves partly from the natural 
lethargy of theorists; it takes time to rewrite theoretical formulations. 
There is also a lag time in publications. Thus, it should not be assumed 
that the theorist's written views are, in fact, necessarily their current 
theoretic positions. Thus, another element in theory development may be 
referred to as the principle of continual development. 

\ater in his talk Guba explains the main reason why Lindheim's 
reconstruction of "him" is so far from his actual position. 
"You of course have no v/ay of knowing about tliat new posture 
from a reading of my published work. Little of my present 
position has found its way into the literature.' 
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Principle 3 : Theoretical formulations are subject 

to continual development or change. (Cor; An adequate 

elucidation of a person's viewpoint must make reference 
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to the time at which he held that view. ) 

In the symposium papers. Wolf indicated that his theories were evol- 
ving and being refined overtime. In speaking of cross-validation processes 
he indicated that "for the last three years, I have continued to refine 
the operationalization of natural inquiry strategies which demonstrate 
powerful applicability to an elaboration of inferences and explanations." 
Guba more clearly than any of the other theorists who participated in this 
seminar, demonstrates how one's beliefs and perceptions change over time. 
"I find myself this morning. . .trying to make the present state of my mind 
appear to be perfectly consistent with my former position. ..even though 
I know it is not." He sympathized with Lindheim's problems in trying to 
characterize "his position". "If... she had some problems in understanding 
what Egon Guba was like, I presently share her dilemma; my mind has changed 
about so many things th^t it is hard for me to recall what opinions I held 
then." In fact, his whole theoretical framework has changed, though he 
is not surprised by this fact. "It should not, after all, be so surprising 
that my thinking has changed a bit over 10 years. ..we all reserve the right 



corollary prescription is associated with this principle. Since 
theories will change over time, theorists should expect that such 
changes are going to occur and in^some way start to reflect upon 
them, map them out and insure that they are going to be as con- 
scious as they can about the changes in their views. The advantage 
of this are twofold. One, of course, is that theorists would have 
more conscious control over or awareness of their own thinking. 
Secondly, the provision of direction and order to the changes as 
they occur may provide more hypotheses about a theory and subse- 
quently, new insights. 
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to change our minds. Hopefully, in this case, it is because my thinking 
g 

has matured." 

The Strategy of Theory Confrontation 

The theoretic positions which have been expounded and presented in 
writing are often modified based upon a variety of confrontation stimuli. 
The view by Guba that his thinking has matured over time is an indication 
that a wide variety of stimuli have forced a reconsideration and reexam- 
ination of his theoretical positions. What are the nature of these con- 
frontation stimuli? 

Confrontation with One's Own Theory . Public disavowals of one's 
prior theoretic views hardly ever take place. Theoretic viewpoints once 
expounded and put forth usually only get modified in the mind of the theori- 
zer and don't get explicated, in writing or verbally, until the theorizer 
is publ icily confronted with his previous theoretic position. Confronta- 
tion with a prior written statement of a theoretic position may take place 
either through someone else questioning elements of the written theory 
(or describing or reacting to the theory) or through one's own re-reading 
and recollecting what it was that ha had to say. 

The presentation and description of theorists views by other parties 
provides perhaps the best opportunity for theorists to reconsider and re- 
construct their own thoughts. When faced with the portrayal of reconstruc- 
tion of one's views, theorists frequently are impelled to consider whether 
the portrayal is correct or not. By doing this, it may lead them to stand 
by their position, to re-interpret the position put forward, or to show 
how evaluation theoretic views have been modified through the years. 



It becomes apparent then that a historical view of the author's 
work is quite useful in examining the development of a theory. 
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work is qu 
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There are distinct advantages in having theoretic positions re- 
constructed by others rather than having the particular theorist 
do it himself. Authors might be inclined to read new meanings into 
their work saying that it expresses vvhat they really meant, or they 
might fail to see the discontinuity between earlier and later writings. 
Portrayers react only to written materials as they might generally be 
interpreted by the public. Portrayal of one's evaluation theory by 
others often leads to stark interpretations— perhaps different from 
what theorists thought they had said or from views currently held. 
This leads us to a consideration of Principle 4. 

Principle 4 : Confrontation with previously written 
theoretic positions is a major basis for modification 
of evaluation theories . 
It might be suggested that we are implying that a linear progres- 
sion takes place in systematic theory development. On the contrary, 
we claim that the nature of theory development as it actually occurs 
within the real world context in evaluation (as well as in other fields), 
is neither systematic nor linear. There is not a constant review and 
constant stimuli from confrontation with prior written theoretic positions. 
In that respect, the 1978 symposium was unique. It presented the opportu- 
nity for various authors to be confronted with their past writings 
and to contrast them to their present views. The theorists were given 
the opportunity to determine "how" these various view related to each 
other and to contrary views of evaluation as well as to ponder the defen- 
sibility of prior written theoretic positions. Note again that certain 
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attitudas were involved here, too. The participants have to be willing 
and committed to a kind of rational discussion of their views. Without 
this, the confrontation with one's prior position would prove useless. 

A corollary prescription is identified-the symposium prescription. 
There ought to be regularly scheduled confrontation-type symposia related 
to evaluation theories. (Not gentle, delicate, symposia, not "survey" 
sessions, not theory application symposia, not information distribution 
sessions, but confrontation symposia that are designed to confront theo- 
rists with specifically reconstructed perceptions of their own work.) 

An examination of the confrontation between Lindheim's statement 
of Guba's views and Cuba's conceptions of his theoretical view proved 
insightful. Confronted with Lindheim's clear statement of his previous 
position and its relationship to the ideas of JNR, Guba tries to under- 
stand what caused him to abandon those ideas. In so doing, he elaborates 
some underlying assumptions that were previously unclear to him and to 
his readers. "I certainly would not find in TNR today corroboration for 
a definition of evaluation as a process of servicing decisions... such a 
formulation is based on assumptions which, although not clear to me in 
1967 are now evident and now found by me to be nonpersuasive. What are 
some of these untenable assumptions? ... that there ij. a decision authority, 

that decisions are made in some explicit way, ... that there is a deci- 
sion time, etc."^^ 

l^What is not clear from this interchange is whether Guba's statement 
of the "discovery" of previously held unsound assumptions was a 
direct response to Lindheim or whether he had become aware of these 
assumptions at a prior time and is merely acknowledging the fact at 
this time. Nevertheless, the confrontation serves as an Wtus 
for his making his (possibly) private reconsiderations public and 
available for scrutiny. 

12 



Confrontation with other theories . In order to understand as fully 
as possible one's own views, it is often helpful to compare and to con- 
trast it with someone else's theories. By noting the differences and 
the similarities among theoretical viewpoints, one is likely to come 
to see the strengths and limitations of each of them. Furthermore, by 
comparing and contrasting, one is more likely to arrive at hypotheses 
for the consolidation of differing viewpoints, for the rejection of 
certain views, or for the invention of new ones to fill obvious gaps. 
Considering another's theory, then, may well help to develop one's own 
theory to bring about a better theory. This is the basis for Principle 5. 
Principle 5 : Confrontation with the written theoretic 
positions on eva l uation is a major bas is for modification 
of evaluation theories. 
This principle is clearly demonstrated in many of the evaluation 
theory papers prevalent in the field. Cronbach and Scriven's work have 
formed the foundation for much of the formulations of evaluation theorists 
at a more specific level. It is obvious that Alkin's early writings on 
evaluation were influenced by Stufflebeam's thinking. 

A further example of reliance on the written theoretic positions of 
other evaluators is provided by statements made by House during the afore- 
mentioned AERA symposium. He noted: "...I set about trying to embody the 
Perelman ideas and other ideas, and to put them in a form that we would 
find recognizable in the evaluation field. I went to the works of some 
of the saints in the f ield—Cronbach and Campbell— and looked at their 
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stuff, knowing that they have been moving in this direction. I used 
their writings as a kind of a jump-off into some of the Perelman rea- 
son reasoning." Furthermore, House notes: "...although Egon was re- 
nouncing his earlier self of years ago, I feel differently. In fact 
that earlier self, the Egon Cuba of 10 years ago, was a very import- 
ant part of what contemporary evaluation is now... We evaluation theo- 
rists have played off his ideas (and of others) to get the ideas we 
now have." 

Principle 5, then, stresses the value to one's theorizing of 
being confronted by the viewpoints of others on the nature of evalua- 
tion. It may come as a surprise to some, however, that the evaluation 
theorist may also prof.t greatly be being confronted with a theory on 
something other than evaluation. It may be u':^fijl for a theorist to 
take a model or method which has proved usefui .■» the inquiry into 
other matters and try to incorporate it into his theory of evaluation. 
For example, a theorist might hypothesize that it would be insightful 
to think of educational evaluation as a kind of legal reasoning and 
evidence gathering or as a kind of decision making. The metaphor of 
the legal system was an important basis for Wolf's writings; and, the 
writings of Braybrooke and Lindblom were an important stimulus to the 
work of Stufflebeam. 

Alternatively, theorists need not take on nor assume the models 
or methods of others, but may compare and contrast them with their own 
theories of evaluation in order to gain further insights into their 
own work. This leads to the formulation of Principle 6: 
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Principle 6 ; Confrontation with wri tten theoretic 
positions on matters other t h<-i ..i evaluat ion is a major 

basis for modification of evaluation theories. [Such 

theoretic positions win He called "external" theori es) 
Several of the presentations in the symposium about which we have 
been reporting illustrate this principle very well. Wolfs reaction 
to the external theory formulation (TNR) is to reaffirm some of his 
own theoretical positions. He maintained that JNR placed too much empha- 
sis on "argumentation without a clear articulation of how argumentative 
skills can benefit the field of educational evaluation." On the other 
hand, he feels that his own judicial evaluation model "allows for the 
juxtaposition of those different viewpoints and different interpretations, 
of evidence" within a structure that provides for cross-validation and 
cross-checking. In this way, "argumentation and persuasion become ill- 
uminating rather than obfuscating." Yet even while denying the value 
of INR's argumentative emphasis. Wolf entertains some new ideas prompted 
by the rhetorical classifications and discussions of JNR: "The essence 
of good evaluation therefore, should not rely upon the skills of argumen- 
tation because we already employ those kinds of skills, often to the 
disservice of clients or program consumers. More useful skills would 
be learning how to understand the significance of contrasting arguments 
and how to challenge or extend inferences and explanations." 

In the same symposium, Rippey noted that he was strongly influenced 
by some of the ideas discussed in INR. He found marked similarities 
between some of his own ideas about transactional evaluation and some 
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of the ideas expressed. 

"I really appreciated Perelman and Olbrects-Tyteca's 
delineation of the differences among impartiality, objectivity, 
partisanship, fanaticism and skepticism. I have always lacked 
patience with evaluators who let projects sink in order to main- 
tain objectivity. ...What they are saying (in TNR) is that the 
evaluator should not be a voyeur. If he is not a member of the 
tean, he will not be able to speak with authority because commit- 
ment and intimacy will be lacking. The objective evaluator is 
as much disqualified for his remoteness as the project director 
for his partisan bias." 

The theoretical concept that had the greatest impact on Rippey 
was the idea of a "community of minds" first, Rippey relates this idea 
to his previous work. "One of the most profound concepts I found in 
TNR was the necessity of establishing a community of minds prior to 
argumentation.... Perhaps the evaluator, who is usually in the position 
of advocate whether he likes it or not, needs to spend more time estab- 
lishing a community of minds committed to argumentation before he begins 
his work. I have previously advocated the use of project monitoring 
condttees made up of both protagonists and the antagnonists of proposed 
inovations." 

Apparently, Rippey believed, prior to the symposium, that any pro- 
tagonist or antagonist should be allowed to be on the connittee. Now 
he feels, that a stronger condition must be satisfied in order for a 
person to be on one of these monitoring committees-basically , he must 
join or be a member of the community of minds. In other words, Rippey 
began to question the composition of monitoring comnittees— an issue 
which had not been fully addressed in his prior theoretical fomulations. 
Rippey regards this issue as a hypothesis or as conjecture to be tested 
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out or to be extended by further work. He noted, "I am anxious to explore 
the effects of the establishment of such a community of minds... perhaps 
there is no point in proceeding before this has been done." The general 
point about evaluation theory development that we are drawing from this 
example is that while reading books in related fields one is often con- 
fronted with an idea which becomes a hypothesis in his own field of 
inquiry. This promising lead deserves further elaboration and the theorist 
prepares to take steps to test out and gather evidence for or against 
its inclusion in the theorists previously established conceptual framework. 

Confrontation with Consolidation and Categorizations . Another 
phase in evaluation theory development becomes possible when theory 
consolidation or theory categorization has taken place. In an earlier 
phase, attempts have been made to re-frame (or define) the structure 
or to elucidate the structure in the field at that time. If there is 
any perceived unity then consolidation takes place; if there isn't any, 
then a categorization may be designed to show differences and dissimilar- 
ities. 

Principle 7 : Confrontation with theory classifications 
either in terms of consolidation into a single theory 
or categorization into types is a major basis for 
modification of evaluation theories. 

In educational evaluation, the late 1960's and early 1970's was 
a period of theory classification. The formation of the Phi Delta Kappa 
National Study Committee on Evaluation (Stufflebeam et al , 1971) was 
an attempt at theory consolidation. Others did not see consolidation 
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as possible and provided categorizations of evaluation theories (Worthen 
and Sanders, 1973; Popham, 1975; Stake, 1974). Attempts at categoriza- 
tion, in turn, prompted some theorists to reconsider and reconstruct 
their formulations. For example, Alkin has shown his reaction to the 
categorizations in articles in the last several years. An examination 
of two of his papers, "A New Role for Evaluators" (1977) and "The Title 
of This Speech is..." (1975) is helpful here. The papers indicate 
his reaction to the categorization of his earlier work as "decision 
oriented" evaluation. While the category title is not inherently dis- 
tasteful or inappropriate, placing him in the category led him to perceive 
important differences. In reacting to decision-oriented evaluators 
and what people say decision oriented evaluation is, ha reveals that 
he regards such an activity as too mechanistic— as appearing to be more 
straight forward. That it in fact is or should be. The category presents 
the illusion that there is always a decision and that evaluation informa- 
tion influences that decision in straight fon^ard, immutable ways. 
He emphasizes that this is not the Way he conceived of evaluation activi- 
ties. (At least, thinking about it now, he doesn't think that is what 
he 'iad in mind. Although, he concedes that if he were to read what 
he wrote he could see how people miqht have thought that his theoretic 
position would fit into that category.) 

So, this phase represents another kind of confrontation— the con- 
frontation with a category system. Hera the theorist regards the cate- 
gories as presenting an over-simplification of theoretic viewpoints 
which tend to emphasize elements of a theory which may no longer be felt 
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appropriate or which emphasize elements in a misleading v/ay. Alkin 
wasn*t reacting primarily to something he wrote, but to the catego- 
rization by others of what he had said. A theorist provoked by such 
categorizations can continue in the theoretical stream which is already 
moving. Having studied the categorizations, he could start off in a 
new direction. 

Robert Stake provides another example of a theorist making a major 
change in theoretical formulation following the categorization stage 
within the evaluation field. All of the categorizations attenpted 
to classify his views based on the countenance model (1957). In res- 
ponse to this categorization phase. Stake began thinking about what 
he calls "responsive evaluation," evaluation which was akin to countenance 
evaluation but more person oriented. 

Field-Based Theory Confrontation . Many evaluation theorists 
actually engage in the conduct of evaluations. Their continuing experi- 
ence at these activities forms another basis for modifying evaluation 
theories. In essence, this may be thought of as a third confrontation 



A look at the status of theory development in evaluation, might 
provide a basis for declaring that there is little hope for theory 
unification. A point that should be made, however is that this is 
not deleterious—there is room for different evaluation views 
usually connoting different purposes. It is not a question of 
different approaches to the attainment of evaluation but rather 
different types of evaluation each really talking about different 
phenomena. The point that we would articulate is that non-unifica- 
tion of theories is appropriate unless it can be demonstrated that 
phenomena described are precisely the same. Let people develop 
their own theories at a level and in their own way and defend them 
with reference to the application domain of their own choice. 
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mode (in addition to confrontation with previous written work and with 
a classification.) We are proposing the principle of field based 

theory confrontation. 

Principle 8 : Confrontation of evaluation theories with 
field based experiences is a major basis for m odification 
of evaluation theories . 
The recent appearance of a book by Michael Patton entitled 
Utilization Focused Evaluation , (1978) provides an example of a theo- 
retical formulation which evolved from field based experiences. Rippey's 
Studies in Transactional Evaluation , (1973) appears also to be heavily 
field based. In actuality, most theories have a strong element of field- 
based-reality and perhaps we are only talking about distinctions in terms 
of degree or extent. House makes the point rather clearly in his remarks 
in the AERA symposium, "...we already are doing many things in evaluation 
that we have no real justification for. I felt rather deliberately that 
I wanted to construct some sort of rationalization to justify many of the 
things that we are already doing. It is allright to try to justify what 
in some cases we are already doing in practice. It is not a hypocritical 
position. The mode of knowing is much more a dialectical mode of knowing." 

The Strategy of Colleqial Dialogue 

Up to this point we might describe a good deal of what we've seen 
as belonging to the social nature of evaluation theorizing. We point 
out how certain social relations with other theorists, both those within 
evaluation and those external to it, can provide a useful basis for the 
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modification of evaluation theories. We have characterized these social 
relations as relations of confrontation. Such a view is inadequate, how- 
ever, for it fails to point out a more positive aspect of evaluation 
theorizing. Beyond the confrontation type activities lies the principle 
of theory development which we will refer to as the strategy of colle- 
gia! dialogue. This theory development activity occurs when two or more 
theorists are jointly working on the same or similar set of ideas. 
In a given instance, the theorists may not be reacting to other people's 
ideas directly but are only working and encouraging each other. Inter- 
action in theory development can serve as a reinforcement and encouragement 
especially when the other participant is someone who was developing 
similar ideas in almost a similar context. 

Principle 9 : Evaluation theory development is often 
fostered by interaction with colleagues who have compati- 
ble points of view . 
An examination of the collegia! stimulation provided by Egon Cuba 
and his role in theory formulation (evaluation theory in particular) is 
instructive. Cuba, while a faculty member at the University of Chicago 
participated with Charles Bidwell in the development of administrative 
theory. Cuba's insights into administrative decision making were an 
important input into the initial joint work in evaluation theory develop- 
ment conducted with Daniel Stufflebeam at Ohio State. Since moving from 
Ohio State, collaborative work at Indiana with David Clark has been fruit- 
ful. And, recently Cuba and his Indiana colleague Wolf have been apparently 
a source of mutual support on matters related to naturalistic inquiry. 
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other instances of the application of this principle are many— in fact 
the joint authorship of this paper by Alkin and Ellett is perhaps a 
prime example of such colleagial support in evaluation theory development. 

The Personal Side of Theory Building 

What has become apparent is that aside from the technical skills and 
social abilities of evaluation theorizing there is a personal dimension 
to theory building. Things like rational capabilities for theory develop- 
ment become mitigated by self-interest, pride or even politics. The reality 
is that human beings do the theorizing so that analyses of evaluation 
theorizing must pay attention to human beings— to the difficulties that 
they have in disavowing what they have believed, to their self interest, 
to their primary personal association, and to political questions. 
Given that such factors exist, it is easy to understand why theoreti- 
cians are constantly warned about letting their bias, prejudices, or 
other irrelevant factors enter into their theorizing' activities. It 
seems that theoreticians must strive to eliminate all of the strictly 
personal aspects of their theorizing. But such a conclusion would be 
too hasty. For, the strictly personal aspects of the theoreticians 
activity, given proper safeguards, need not be an impediment to viable 
theorizing but may well faster it. The role of personal hunch, imagina- 
tion, and intuition may serve the theorist in constructive ways. Given 
that the ideas and hypotheses suggested by the person's imagination (or 
intuition) are subjected to the proper tests, the person's imagination 
and intuition may be a profitable source of good ideas and play a major 
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role in the modification of evaluation theory. This leads us to the 

formulation of principle IQ, 

Principle 10 : Evaluation Theory development is conditioned 
by human beings, their associations, their motivations, in- 
tuitions and imaginations, (We shall refer to these as 
personal factors «) 

Some years ago Daniel Stufflebeam (under the auspices of Phi Delta 
Kappa) convened a meeting of leading evaluation theorists to attempt to 
define a unified theory of evaluation. The work of this committee 
eventuated tn a book on evaluation (Stufflebeam, et al , 1971) but failed 
to effect the kind of theory consolidation that had been envisaged by the 
sponsors of the committee- In part, this was a function of the choice 
of participants to the group; In part also the declination to participate 
by other advocates of different theoretical points of view may have been 
a strong factor in the failure to obtain better theory consolidation. 
The pattern of personal associations, self interests and motivations may 
well have been a major factor in that episode of evaluation theory develop- 
ment. 

As another example of the Importance of personal factors in evalua- 
tion theory deve^opnnent, we should note the recent Phi Delta Kappa study 
of evaluation theory currently conducted by Daniel Antonoplos and William 
Gephart is attempting to categorize evaluation theories based primarily 
on the pattern of personal associations and Interactions of particular 
evaluation theorists. 

Another illustration of evaluation theorizing which has been condi- 
tioned by personal factors— in this case, a person's intuitions— was found 
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in the previausly discussed AERA symposium. Rippey's remarks Illustrate 
how his reading of M resulted in his having Cnew) hunches or intuition 
of a plausible way of developing confidence testing. "Confidence testing 
seems to me . and seemed to me even on my first run through TNR to be a 
suitable methodology for studying evaluation as argumentation." Rippey's 
reactions need not have been shared by any other member of the symposium. 
The intuitions represent a personal response by him to M- 0^ ^o^^-se, 
such intuitions (or hunches) need to be tested out. But our point here is 
to emphasize the positive role which a person's hunches and intuitives 
can play in developing a defensible evaluation theory. 

Combined Strategies 

Theory reformulations take place based on combinations of factors- 
no single principle acts in isolation of others. That is. for example, 
a theory reformulation may be partially field based. Alkin feels that 
he was willing to accept intuitions from the field which he had pre- 
viously rejected (based on false notions of what he felt was appropriate 
scholarship and admissable data to theory development.). In part, his 
willingness to accept field data stemmed from his dissatisfaction with the 
previous mentioned categorizations. The combination of confrontation with 

a categorization and confrontation with field practice (field based intui- 

.12 

tion) led- to a totally new theory building thrust for him. 



12see: Alkin. M. C. Daillak..R. L.. and White. P'S- WhUe. 
Evaluations . Library of Social Research. #76. Beverly Hills. Sage 
Publications., (1979). 
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Final Remarks 

In this essay, we have been trying to shew that evaluation theo- 
rizing should be thought of as a human activity which is governed by 
certain canons and strategies. In order to develop a viable theory, 
a theoretician uses these principles to help guide his theorizing. 
We believe that the benefits of conceiving of evaluation in this way 
are theoretical, practical, and pedagogical. We hope our initial 
effort will provide the grounds for further insights and future devel 
ment of the conceptualization of evaluation as a human activity. 
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